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Abstract. The paper deals with the facts obtained from TIMSS 2003 (Trends in International 
Mathematics and Science Study). This international comparative study, which includes 47 participant 
countries worldwide, explores dependence between eighth grade students’ achievement in the areas of 
mathematics, physics, chemistry, biology and geography, and basic characteristics of context for 
teaching and learning in school and at home. In the sense, different options of using modern 
technology equipment and its influence to students’ achievement are explored in the TIMSS 2003 
assessment. The main topic of the paper is using computers and calculators in teaching and its 
implications to students’ overall achievement at the end of primary school education. The TIMSS 2003 
international overall results in this area show that using computers in teaching doesn’t significantly 
contribute to better students’ achievement in the field of mathematics and also show some level of 
significant influence on students’ achievement in the field of science. Moreover, the results show that 
using calculators in mathematics teaching improve overall students’ achievement. Connectedness 
between using computers/calculators and students’ achievement is especially explored and presented in 
the frame of students’ sample in four countries, the United States, the Netherlands, Bulgaria and 
Serbia. 
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The main results of the TIMSS 2003 assessment in the fields of students’ mathematics and 
science achievement in the eighth grade and how they were essentially affected by some 
characteristics of teaching and learning mathematics and science, especially by using 
computers and calculators in teaching, represent one of the attractive research topics in the 
TIMSS 2003 assessment. In order to assist appropriate analyzing of students’ achievement in 
the fields of mathematics and science, some contextual information are presented in the paper, 
and those information were obtained from the mathematics and science teachers, on the base 
of gathered extensive data on curricula, and teaching and learning mathematics and science in 
primary school. 

The initial idea was to confirm presence and intensity of some characteristic 
influences on students’ achievement inside the Serbian eighth grade students’ sample, in the 
area of using computers and calculators in mathematics and science teaching. For instance, we 
could particularly confirm that in the Serbian eighth grade students’ sample there is difference 
in mathematics achievement between two groups of students, students that use and students 
that don’t use computers in the mathematics teaching. Furthermore, we have concluded that it 
can be very interesting to apply a cross-country comparison of these kinds of confirmation of 
influences on students’ achievement, which arise from mathematics and science teaching and 
learning characteristics. An appraisal had made that it is interesting to compare students’ 
achievement influences that appear in high developed countries, in comparing to the similar 
kind of influences that appear in countries in process of social and economic transition. Based 
on these criteria of designing comparison model, the analyses included investigating of 
influences on students’ mathematics and science achievement in four TIMSS 2003 participant 
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countries: the United States, the Netherlands, Bulgaria and Serbia. Cross-country comparisons 
of the TIMSS data are often made in order to establish differences in factors affected students’ 
achievement. 



Methodology 

One of the basic standpoints is general positive attitude towards opportunities of teaching and 
learning in the direction of advancing development of student cognitive abilities and skills. In 
the sense, the research topics were mostly targeted to the following: use of computers and 
calculators in mathematics and using computers in science teaching, and their significance to 
students’ achievement in the areas of mathematics and science. We have applied specific 
combined model of analysis separate variables in the paper, which includes statistical analysis 
and content analysis as well. For example, when we discussed obtained statistical indexes, we 
didn’t make only stay at the level of statistically oriented type of analysis. In order to make 
more complete and thorough analysis of the chosen topics, we have tried to give appropriate 
connections toward broader context that determines applying of treated teaching methods and 
activities, through theoretically -based content analysis. 

We have formulated the next research question in the study: What are the levels of 
influence of using computers and calculators in teaching and learning mathematics and 
science to students ’ achievement in the field of mathematics and science across four-country 
sample of eighth grade students! 

Table 1: Students’ average achievement in mathematics across countries 



Country 


N 


Average 

Achievement 


Standard Error 


Netherlands (NLD) 


3065 


536.27 


3.8 


United States (USA) 


8912 


504.37 


3.3 


Serbia (SCG)* 


4296 


476.64 


2.6 


Bulgaria (BGR) 


4117 


476.17 


4.3 


Intern. Average 




467 


0.5 



* In the TIMSS 2003 unique country abbreviation for Serbia was SCG, but the sample 
consisted only from Serbian primary schools. 



The overall eighth grade students’ achievement results in mathematics are given in the Table 
1, and for the area of science in the Table 2, as well as number of students in national students’ 
samples. Total number of students in four-country sample is 20390 students. In the following 
tables numbers of students are mainly different regarding to the data from tables 1 and 2. That 
is because of the fact that numbers of students belonging to the sub-group of teachers vary 
depending on number of teachers that gave answers on some questions. 

Table 2: Students ’ average achievement in science across countries 



Country 


N 


Average 

Achievement 


Standard Error 


Netherlands (NLD) 


3065 


538.44 


3.1 


United States (USA) 


8912 


525.31 


3.1 


Bulgaria (BGR) 


4117 


478.96 


3.8 
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Intern. Average 




474 


0.6 


Serbia (SCG) 


4296 


468.48 


2.5 



The national defined population of primary schools for the TIMSS 2003 sample in Serbia 
consisted of 1100 primary schools. The multistage cluster sample model in Serbia included 
149 primary schools, proportionally selected by cluster model of sampling from each of the 
stratified region. School-level exclusions consisted of schools near the Kosovo region, special 
education schools and very small schools. An explicit stratification was not made but there 
was implicit stratification by the region (Central Serbia, Belgrade and Vojvodina) and by 
urban-rural criterion. There were a total of six implicit strata. After all kinds of the applied 
exclusions, the Serbian sample was represented by 4296 eighth grade students (2206 boys and 
2090 girls), 177 mathematics teachers and 702 science teachers. The mean age of tested eighth 
grade students in Serbia was 14.9 years. 

In our research some statistical procedures were applied and measures were 
obtained, in the purpose of exploring some characteristics of the quantitative and qualitative 
dependences between chosen variables analyzed in the research. We had connected variables 
from four countries’ BTM and BTS files (data from eighth grade mathematics and science 
teachers’ questionnaire) and BSA files (eighth grade students’ achievement data). Merging of 
files were done using student-teacher linkage files (BST). These statistical procedures and 
measures were applied in order to make proper descriptions, on a statistical way, and 
characteristics of the chosen group of the mathematics teachers and their students in each of 
four countries, as well as some characteristics cross-country oriented. From variety of 
statistical procedures, we have included TIMSS mean scale scores, per cent measure, 
statistical significance, Fisher’s coefficient, etc. All estimations of students’ achievement in 
the mathematics are made by calculating with mathematics plausible value” and 
science plausible value”, which exist for each student in the students’ BSA files, as well as in 
the files merged using student-teacher linkage files. DPC IDB Analyzer software was used in 
carrying out each of defined analysis in the research. Using of this software is associated with 
the SPSS software, and both accompanied enabled appropriate approach to the TIMSS 
International Data Base. 



Computers and calculators in mathematics teaching 

The TIMSS 2003 testing procedures permit of calculators using in the course of testing, both 
in sectors of mathematics and of science (Mullis et al., 2004). In the TIMSS assessments there 
are some kinds of confirmed dependences between calculator using policy and students’ 
achievement in the mathematics teaching. It might be hypothesized that group of students 
which were not allowed to use calculators in the mathematics teaching can achieve better 
results than a group of students were allowed to use calculators. The reason for this can be 
found in the fact that students not allowed to use calculators had to master all needed abilities 
and skills in the area of calculating, including development of some concepts related to the 
used mathematical procedures and operations. Moreover, it may be a topic of discussion 
whether it really occurs in the mathematics teaching and learning activities, or students, 
instead of advancing needed concepts in the area of calculating, only have had opportunity to 
exercise mechanically operations of calculating, without deeper approaching to their internal 
essence and interconnectedness. It had been suggested by Reynolds & Farrell that the early 
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introduction of calculators, and their too frequent use, is one of the reasons for the relatively 
poor performance of students in England in mathematics (Keys, 1999) 

Table 3: Calculator availability on the mathematics lessons: cross-country 



Options 


M 


Total N 


Percentage 


S.D. 


All 


522.74 


7770 


49.69 


76.86 


Most 


505.46 


2883 


18.44 


79.36 


About half 


469.31 


1049 


6.71 


85.04 


Some 


479.45 


3589 


22.95 


82.57 


None 


457.78 


346 


2.21 


72.01 


TOTAL 


504.59 


15637 


100.00 


81.86 


F=1006.474; df=l;p=.000 



In the Table 3 can be seen that the best achievement in mathematics belongs to the sub-group 
of students in four-country sample, whose teachers reported of calculators’ availability for all 
students in the mathematics teaching. This fact in not in accordance to the hypothesized 
expectation that students which don’t permit to use calculators are enabled to develop some 
abilities and skills in the area of calculating and consequently achieve better results in 
mathematics assessment. A significant negative relationship between the frequency of 
calculator using and students’ mathematics achievement in Japan was identified in the TIMSS 
1999 and non-significant relationship for the students’ achievement in the United States 
(House, 2002). In one other study (Keys, 1999), based on the TIMSS 1995 study results, it 
was confirmed that across countries there was very little association between the extent of 
calculator use and mean mathematics score (Spearman’s rho = -0.17). 

Table 4: Calculator availability on the mathematics lessons: separate countries 



Country 




All 


Most 


About 

half 


Some 


None 


BGR 


M 


522.74 


505.46 


469.31 


479.45 


457.78 


S.D. 


76.86 


79.36 


85.04 


82.57 


72.01 


NLD 


M 


554.01 


509.15 


432.62 


- 


- 


S.D. 


62.42 


64.74 


63.17 


- 


- 


USA 


M 


511.82 


508.35 


466.34 


491.82 


482.73 


S.D. 


78.16 


76.28 


75.76 


75.34 


50.61 


SCO 


M 


461.93 


473.17 


468.42 


476.24 


490.76 


S.D. 


84.22 


84.13 


91.32 


90.01 


88.47 



In this case, the variable had chosen that examine using computers as a means for the purpose 
of “discovering mathematics concepts and principles”, considering significance of 
understanding mathematics concepts and principles for better students’ performance in the 
mathematics. As it is shown from the next table, computers are rarely used in the mathematics 
teaching, across four countries’ eighth grade. It is scarcely over 5% of the mathematics 
teachers reported of using computers in this purpose. 

It was interesting to try exploring the connection between using computers in the 
mathematics teaching and sub-group students’ mathematics achievement, and also what is 
nature of this connection, if it is existed in the field of mathematics. One of the items in the 
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mathematics teachers’ questionnaire is formulated through the next question: “In teaching 
mathematics to the TIMSS class, how often do you have students use a computer for the 
following activities?” There were four activities for the teachers’ appraisal and the most 
interesting is variable of “discovering mathematics principles and concepts”. 

Table 5: Using computers for “discovery mathematics 
concepts and principles cross-country 



Options 


M 


N 


Percentage 


S.D. 


Every or almost every 
lesson 


480.58 


105 


2.15 


86.08 


About half the lessons 


491.04 


153 


3.13 


95.55 


Some lessons 


510.19 


2290 


46.78 


82.46 


Never 


501.30 


2347 


47.95 


77.49 


TOTAL 


504.69 


4895 


100.00 


80.85 


F=.047; df=l; p=.828 



Results in the previous table have shown that almost 95% of students across four-country 
sample use computers only in “some lessons” or “never”. This important fact points out that 
computers mostly have not adequate applying in the mathematics teaching and learning, but 
the most successful sub-group of students in the four-country sample are this one which were 
enabled to use computers in process of discovering mathematics concepts and principles. 

Table 6: Using computers for “discovery mathematics 
concepts and principles separate countries 



Country 




Every or almost 
every lesson 


About half the 
lessons 


Some lessons 


Never 


BGR 


M 


- 


- 


534.05 


478.95 


S.D. 


- 


- 


80.07 


72.38 


NLD 


M 


- 


- 


550.25 


537.94 


S.D. 


- 


- 


65.92 


60.21 


USA 


M 


408.39 


495.75 


502.54 


497.38 


S.D. 


54.36 


94.34 


80.69 


78.45 


SCO 


M 


509.45 


461.41 


436.02 


460.41 


S.D. 


79.36 


100.09 


102.07 


84.59 



The Table 6 shows that, in the cases of Bulgaria’s and the Netherlands’ mathematics teachers’ 
samples, there are no teachers reported about using computers for discovering mathematics 
concepts and principles in the options of “every or almost every lessons” and of “about half 
the lessons”, but also in other two samples of the mathematics teachers there are low 
percentage of teachers chosen these options. Generally, computers have used rarely in the 
mathematics teaching across countries, as it is the case for using computers in the area of 
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helping student to discover and understand some mathematics concepts and principles. In an 
analysis of the factors affecting students’ achievement in the field of mathematics, applied for 
Hong Kong, Cyprus and the United States students’ samples in the TIMSS 1999 study it was 
confirmed that the highest means generally belong to students who never used computers, and, 
in addition, the lowest mathematics score average belonged to the students who used 
computers for most of their lessons (490.28) (Papanastasiou, 2002). The results of this analysis 
show that using computers frequently in the mathematics teaching does not necessarily 
increase students’ mathematics achievement. 



Computers in science teaching 

There is a general tendency of increasing of using computers in teaching and learning science. 
In many fields of science teaching it is convenient to use different kind of software, in order to 
improve overall quality of teaching and learning. It is especially important for better and 
deeper understanding of crucial scientific concepts and laws. 

In the TIMSS 2003 science teachers’ questionnaire there are several questions 
devoted to using computers for teaching and learning science. The most significant is question 
26, with following formulation: “In teaching science to the TIMSS class, how often do you 
have students use a computer for the following activities?” And, there are the options: (a) “do 
scientific procedures and experiments”, (b) “study natural phenomena through simulations”, 
(c) “practice skills and procedures”, (d) “look up ideas and information”, and (e) “process and 
analyze data”. I was interesting to explore indexes across first two options inside this question. 
There is also a general question about using computers in science teaching (25. a. “Do students 
in the TIMSS class have computers available to use during their science lessons?”) and about 
access to the Internet (25.b.“Do any of the computers have access to the Internet?”) (see tables 
7 and 8). 



Table 7: Using computers: separate countries 



Country 


“Yes” (%) 


M 


“No” (%) 


M 


BGR 


16.04 


484.55 


83.96 


473.22 


NLD 


39.16 


541.34 


60.84 


539.05 


USA 


74.38 


531.36 


25.62 


516.79 


SCO 


14.07 


464.85 


83.93 


468.62 



In the Table 7 we can see percentage of science teachers across the countries which reported 
about using computers in science teaching. Based on the facts obtained, it can be concluded 
that Serbian eighth grade students have the least opportunities to use computers in science 
teaching (14.07%), and the least percentage (Table 8: 42.47%) of computers used in Serbia are 
connected to the Internet, regarding to the other three countries. Circumstances about using 
computers in science teaching and access to the Internet are very similar in Bulgaria, such as 
in Serbia. It can be supposed that situation in this area is much better in the present time. 
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because Serbian Ministry of Education and Sport had supplied primary and secondary schools 
with great number of computers in the meantime. 



Table 8: Internet access: separate countries 



Country 


“Yes” (%) 


M 


“No” (%) 


M 


BGR 


68.69 


485.40 


31.31 


482.33 


NLD 


94.49 


539.53 


5.51 


533.85 


USA 


96.85 


529.50 


3.15 


529.21 


SCO 


57.53 


462.19 


42.47 


477.26 



The previous table shows the fact that the sample of primary schools from the United States 
has the biggest per cent of computers used in science teaching with the Internet access 
(96.85%), and then the sample of the Netherlands (94.49%). The facts are in accordance to a 
level of general socio-economic power and wealth in these two countries. It is interesting that 
difference in students’ achievement across categories of “yes” and “no” is the lowest just in 
the case of the United States sample of eighth grade students (M for “yes” subgroup is 529.50 
and for “no” subgroup is 529.21). It is unexpected that the Serbian eighth-graders have better 
average science achievement in “no” subgroup. 

Table 9: Using computers for “do scientific procedures and experiments 

cross-country 



Options 


M 


N 


Percentage 


S.D. 


Every or almost every 
lesson 


451.29 


292 


2.59 


78.20 


About half the lessons 


505.07 


242 


2.14 


79.41 


Some lessons 


525.46 


4617 


40.88 


79.87 


Never 


515.67 


6144 


54.40 


81.35 


TOTAL 


517.78 


11295 


100.00 


81.50 


F=25.043; df=l; p=.000 



Overall students’ achievement in science across four-country sample, in the variable devoted 
to exploring of using computers for exercising some of scientific procedures and experiments, 
shows the fact that the best achievement is discovered in category “some lessons”. It would be 
hypothetically unexpected to conclude that using computers on “every or almost every lesson” 
can enable lower students’ achievement in science. Problem of using computers is more 
complex and it implies a necessity to explore concrete ways of their using in teaching and 
learning science. In the TIMSS 2003 it was not completely done, with the applied structure of 
variables. 



Table 10: Using computers for “do scientific procedures and experiments” : 

separate countries 
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Country 




Every or almost 
every lesson 


About half the 
lessons 


Some lessons 


Never 


BGR 


M 


- 


501.08 


495.20 


481.02 


S.D. 


- 


71.21 


88.35 


84.48 


NLD 


M 


- 


520.99 


538.09 


541.98 


S.D. 


- 


29.14 


60.66 


55.71 


USA 


M 


450.69 


510.31 


533.36 


531.02 


S.D. 


59.68 


94.20 


79.46 


78.32 


SCO 


M 


451.43 


457.84 


479.90 


464.17 


S.D. 


82.06 


71.78 


81.38 


93.71 



The Table 10 shows that, in the cases of Bulgaria’s and the Netherlands’ science teachers’ 
sample, there are no teachers reported using computers for “do scientific procedures and 
experiments” in the option of “every or almost every lessons”. However, on the base of the 
results inside the variable we cannot make some precise conclusion and similar case we have 
in the next variable, which results are presented in the tables 11 and 12. These variables 
explore presence of some activities of using computers in science teaching, in a common 
sense. There is no adequate information about contents and quality of these activities, as well 
as no information about contents and quality of computer software used, as the base of their 
realization in science teaching. 

Table 11: Using computers for “study natural phenomena through simulations”: 

cross-country 



Options 


M 


N 


Percentage 


S.D. 


Every or almost every 
lesson 


460.71 


306 


2.73 


79.39 


About half the lessons 


532.10 


208 


1.86 


79.76 


Some lessons 


528.19 


4896 


43.72 


77.32 


Never 


512.68 


5789 


51.69 


82.64 


TOTAL 


518.41 


11199 


100.00 


81.15 


F=.13;df=l;p=.000 



From the Table 1 1 we can see that the best results are achieved in subgroup of students, whose 
teachers have chosen the category of “about half the lessons”. Thus, on the base of these 
results, it can be concluded that using computers for “study natural phenomena through 
simulations” is of greater importance for students’ achievement in science, than using 
computers for “doing scientific procedures and experiments”. 



Table 12: Using computers for “study natural phenomena through simulations” : 

separate countries 
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Countiy 




Every or almost 
every lesson 


About half the 
lessons 


Some lessons 


Never 


BGR 


M 


- 


468.12 


511.66 


479.38 


S.D. 


- 


43.81 


91.28 


81.03 


NLD 


M 


- 


- 


541.88 


539.61 


S.D. 


- 


- 


55.67 


57.60 


USA 


M 


463.00 


553.56 


533.65 


528.98 


S.D. 


68.10 


72.49 


77.24 


82.34 


SCG 


M 


459.64 


457.84 


473.65 


464.62 


S.D. 


84.24 


71.78 


83.02 


92.81 



The Table 12 shows that, in the cases of Bulgaria and the Netherlands’ science teachers’ 
samples, there are no teachers reported about using computers for “do scientific procedures 
and experiments”, in the option of “every or almost every lessons” (Bulgaria, the Netherlands) 
and in the option “about half the lessons” (the Netherlands). Here is also shown the fact that 
results in the Serbian students’ sample give us opportunity for good prediction in some new 
cases of exploration of using computers in the area named as “study natural phenomena 
through simulations”, on the base of the present conception and quality of science curricula for 
the Serbian primary school. 



Discussion 

Achievement results and their dependence of some contextual factors of teaching and learning 
mathematics and science through chosen subset of variables explored across four-country 
sample and across countries separately show that there are differences in influence of some 
factors of teaching and learning mathematics and science across four countries. In addition, 
some variables chosen for analysis of achievement influence are not reliable predictors of 
students’ success in the fields of mathematics and science. In some cases they show 
predictable measures, but in some other cases these results are unexpected. 

The using computers in the mathematics and science teaching, as teachers from 
four countries reported, imply mainly better results of students’ achievement in mathematics 
and science. But, there are also some unpredictable results, which don’t enable withdrawing of 
more complete conclusions in the area. 

The problem of using calculators in mathematics teaching is not properly covered 
with the chosen items and their variables in the mathematics teachers’ questionnaire. For 
example, when it was intention to examine by some questionnaire items purpose of using or 
non-using calculators in the mathematics teaching, it was intended to explore the most general 
facts in this area by chosen items. It was objectively supposed that sub-group of students 
which were not permitted to use calculators in the mathematics teaching can achieve better 
results in the field of mathematics, because they have had opportunity to advance some 
calculating abilities and skills. However, there were no items which could explore if there are 
developed concepts in the area of calculating, following of calculators’ non-using. Given 
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students’ achievement results across countries show broader diversity and it is hard to fully 
explain them on the base of chosen variables as predictors of students’ achievement. It 
represents one of the shortages seen in the structure of the mathematics teachers’ 
questionnaire, as well as in secondary analysis generally. 

Across four-country eighth grade students’ sample some structure of students’ 
achievement is made, which varies in accordance to the separate conditions in each chosen 
TIMSS 2003 participant country, in the area of using computers and calculators in the 
mathematics and science teaching. The facts obtained from the TIMSS 2003 show some 
differences which arise from levels of general socio-economic power and wealth in the United 
States, the Netherlands, Bulgaria and Serbia. 

Note. This article is a result of the project »Education for knowledge-based society« No 
149001 (2006-2010), financially supported by the Ministry for Science and Environmental 
Protection, Republic of Serbia. 
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